We apply topological data analysis (TDA) to speech classification problems and to the introspection of a pretrained speech model, HuBERT. To this end, we introduce a number of topological and algebraic features derived from Transformer attention maps and embeddings. We show that a simple linear classifier built on top of such features outperforms a fine-tuned classification head. In particular, we achieve an improvement of about $9\%$ accuracy and $5\%$ ERR on four common datasets; on CREMA-D, the proposed feature set reaches a new state of the art performance with accuracy $80.155$. We also show that topological features are able to reveal functional roles of speech Transformer heads; e.g., we find the heads capable to distinguish between pairs of sample sources (natural/synthetic) or voices without any downstream fine-tuning. Our results demonstrate that TDA is a promising new approach for speech analysis, especially for tasks that require structural prediction.
translated by 谷歌翻译
数据表示的比较是一个复杂的多个方面问题,尚未享受完整的解决方案。我们提出了一种用于比较两个数据表示的方法。我们介绍了表示拓扑分歧(RTD),测量在两点云之间的多尺度拓扑中的异常相同,在点之间的一对一的对应关系。数据点云被允许位于不同的环境空间中。RTD是少数基于TDA的实用方法之一,适用于真实机器学习数据集。实验表明,提议的RTD同意对数据表示相似性的直观评估,对其拓扑结构敏感。我们申请RTD在各种问题的计算机视觉和NLP域中获得神经网络表示的见解:培训动力学分析,数据分配转移,转移学习,集合学习,解剖学评估。
translated by 谷歌翻译